Papers with long-form question answering
SEMQA: Semi-Extractive Multi-Source Question Answering (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent proposed long-form question answering systems have shown promising capabilities, but attributing and verifying their generated abstractive answers can be difficult. |
| Approach: | They propose a task that summarises multiple sources in a semi-extractive fashion . they create a dataset with human-written semi-extractive answers to natural and generated questions . |
| Outcome: | The proposed task summarizes multiple sources in a semi-extractive fashion and produces fine in-line attributions by-design that are easy to verify, interpret, and evaluate. |
LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback (2024.findings-naacl)
Copied to clipboard
Wenda Xu, Daniel Deutsch, Mara Finkelstein, Juraj Juraska, Biao Zhang, Zhongtao Liu, William Yang Wang, Lei Li, Markus Freitag
| Challenge: | Recent large language models (LLMs) are leveraging human feedback to improve their output quality. however, human feedback is costly to collect, especially at inference time when the model provides new, unseen input. |
| Approach: | They propose an inference-time optimization method to refine large language models' output based on fine-grained feedback to pinpoint defects and guide iterative refinement . |
| Outcome: | The proposed method consistently outperforms baseline approaches on three text generation tasks, including machine translation, long-form question answering, and topical summarization. |
Groundedness in Retrieval-augmented Long-form Generation: An Empirical Study (2024.findings-naacl)
Copied to clipboard
| Challenge: | a significant portion of correct answers remain compromised by hallucinations in large language models. |
| Approach: | They examine whether every generated sentence is grounded in retrieved documents or the model’s pre-training data. |
| Outcome: | The findings highlight the need for more robust mechanisms in large language models to mitigate the generation of ungrounded content. |
Modeling Exemplification in Long-form Question Answering via Retrieval (2022.naacl-main)
Copied to clipboard
| Challenge: | Exemplification is a process by which writers explain or clarify a concept by providing an example. |
| Approach: | They propose to use a partially-written answer to query a large set of human-written examples extracted from a corpus to determine exemplification quality. |
| Outcome: | The proposed model is able to retrieve human-written examples from a corpus and show that it is more relevant than state-of-the-art models. |
FinTextQA: A Dataset for Long-form Financial Question Answering (2024.acl-long)
Copied to clipboard
| Challenge: | Existing financial question answering datasets lack scope diversity and question complexity. |
| Approach: | They propose to use a dataset for long-form question answering in finance to evaluate QA systems. |
| Outcome: | The proposed dataset includes 1,262 high-quality, source-attributed QA pairs extracted and selected from finance textbooks and government agency websites. |
Hurdles to Progress in Long-form Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Long-form question answering (LFQA) involves retrieving documents relevant to a given question and using them to generate a paragraph-length answer. |
| Approach: | They propose a long-form question answering system that relies on sparse attention and contrastive retriever learning to achieve state-of-the-art performance on the ELI5 LFQA dataset. |
| Outcome: | The proposed system tops the public leaderboard on the ELI5 LFQA dataset, but it has several troubling issues. |
LongForm: Effective Instruction Tuning with Reverse Instructions (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on instruction tuning relies on expensive human annotation and crowd-sourced datasets with alignment issues. |
| Approach: | They propose a method to generate instructions via LLMs from human-written corpus examples using reverse instructions. |
| Outcome: | The proposed method outperforms larger language models without instruction tuning on tasks such as story/recipe generation and long-form question answering. |
Revisiting Sentence Union Generation as a Testbed for Text Consolidation (2023.findings-acl)
Copied to clipboard
| Challenge: | In order to acquire knowledge on a new subject, it is often necessary to consult multiple sources of written information. |
| Approach: | They propose to revisit the sentence union generation task as an effective well-defined testbed for assessing text consolidation capabilities. |
| Outcome: | The proposed evaluation protocol includes human and automatic evaluations. |
ALaRM: Align Language Models via Hierarchical Rewards Modeling (2024.findings-acl)
Copied to clipboard
| Challenge: | Current alignment approaches struggle with inconsistency and sparsity of human supervision signals. |
| Approach: | They propose a framework modeling hierarchical rewards in reinforcement learning from human feedback (RLHF) it integrates holistic rewards with aspect-specific rewards to enhance alignment of large language models with human preferences. |
| Outcome: | The proposed framework improves the alignment of large language models with human preferences by integrating holistic rewards with aspect-specific rewards. |
PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to LFQA are limited to information-seeking questions . however, users' questions in the real-world may often mislead language models (LMs) to output misinformation. |
| Approach: | They propose a unified approach capable of handling any type of information-seeking question. |
| Outcome: | The proposed approach can handle any type of information-seeking question. |
Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent approaches to language model alignment assume homogeneous human preferences, but actual human preferences vary widely and are hard to satisfy with a single language model. |
| Approach: | They propose an RL-free extension of Direct Preference Optimization (DPO) that folds language modeling directly into reward modeling and trains language models as collective reward models that combine all objectives with specific weights. |
| Outcome: | The proposed method matches or outperforms existing methods in safety alignment and long-form question answering. |
Retrieving Contextual Information for Long-Form Question Answering using Weak Supervision (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing retrievers for long-form question answering are optimized for information that directly targets the question, missing out on contextual information. |
| Approach: | They propose to use weak supervision techniques to optimize retrieval for contextual information. |
| Outcome: | The proposed techniques improve the end-to-end QA performance on a conversational QA dataset. |
Generate, Discriminate, Evolve: Enhancing Context Faithfulness via Fine-Grained Sentence-Level Self-Evolution (2025.findings-acl)
Copied to clipboard
Kun Li, Tianhua Zhang, Yunxiang Li, Hongyin Luo, Abdalla Mohamed Salama Sayed Moustafa, Xixin Wu, James R. Glass, Helen M. Meng
| Challenge: | Existing methods to improve context faithfulness in large language models are either inadequate or overlook the potential for self-improvement. |
| Approach: | They propose a framework that enhances context faithfulness through fine-grained sentence-level optimization. |
| Outcome: | Experiments on ASQA and ConFiQA datasets show that GenDiE surpasses baselines in faithfulness and correctness and exhibits robust performance for domain adaptation. |